Fix minimum version of cublas for grouped gemm #2631

pggPL · 2026-01-28T17:27:48Z

Description

Fixes build issue introduced with #2502 - that PR had incorrect minimal version of cublas and it results in build fails on some containers.

Type of change

Documentation change (change only to the documentation, either a fix or a new content)
Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Infra/Build change
Code refactoring

Checklist:

I have read and followed the contributing guidelines
The functionality is complete
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes

Signed-off-by: Pawel Gadzinski <[email protected]>

pggPL · 2026-01-28T17:28:47Z

/te-ci

greptile-apps · 2026-01-28T17:30:29Z

Greptile Overview

Greptile Summary

This PR corrects the minimum cuBLAS version requirement for grouped GEMM from 13.1.0 to 13.2.0, fixing build failures introduced in PR #2502.

Changes:

Updated compile-time version checks from CUBLAS_VERSION >= 130100 to CUBLAS_VERSION >= 130200
Updated runtime version checks from cublas_version() >= 130100 to cublas_version() >= 130200
Updated error messages and comments to reference cuBLAS 13.2+ instead of 13.1+
Fixed namespace qualification for cuda:: function calls to use transformer_engine::cuda::

Issue Found:

Line 642 in cublaslt_grouped_gemm.cu error message still says "upgrade to CUDA 13.1 or newer" but should say "CUDA 13.2 or newer"

Confidence Score: 4/5

Safe to merge once the error message at line 642 is corrected
The PR correctly fixes the version check from 13.1.0 to 13.2.0 across all code paths, but contains one inconsistent error message that references CUDA 13.1 instead of 13.2
transformer_engine/common/gemm/cublaslt_grouped_gemm.cu line 642 needs correction

Important Files Changed

Filename	Overview
transformer_engine/common/gemm/cublaslt_grouped_gemm.cu	Updated version checks from 13.1.0 to 13.2.0, but error message at line 642 still references CUDA 13.1
transformer_engine/common/include/transformer_engine/gemm.h	Updated documentation to reference cuBLAS 13.2+, but still mentions CUDA 13.1+ which may be inconsistent

Sequence Diagram

sequenceDiagram
    participant User as User Code
    participant API as nvte_grouped_gemm
    participant Check as Version Check
    participant cuBLAS as cuBLAS 13.2+
    
    User->>API: Call nvte_grouped_gemm()
    API->>Check: Verify CUBLAS_VERSION >= 130200
    alt Compile-time check fails
        Check-->>User: Error: Requires cuBLAS 13.2+
    else Compile-time check passes
        API->>Check: Verify runtime cublas_version() >= 130200
        alt Runtime check fails
            Check-->>User: Error: Runtime cuBLAS too old
        else Runtime check passes
            API->>Check: Verify SM arch >= 100 (Blackwell)
            alt Architecture check fails
                Check-->>User: Error: Requires Blackwell or newer
            else All checks pass
                API->>cuBLAS: Execute grouped GEMM
                cuBLAS-->>API: Result
                API-->>User: Success
            end
        end
    end

greptile-apps

_{3 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

pggPL · 2026-01-28T18:19:57Z

/te-ci jax L0

Signed-off-by: Pawel Gadzinski <[email protected]>

pggPL · 2026-01-29T00:04:20Z

/te-ci pytorch

greptile-apps

_{3 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

zhongbozhu · 2026-01-29T00:11:27Z

transformer_engine/common/gemm/cublaslt_grouped_gemm.cu

+  NVTE_CHECK(transformer_engine::cuda::sm_arch(current_device) >= 100,
             "nvte_grouped_gemm requires Blackwell (SM100) or newer architecture.");
-  NVTE_CHECK(cuda::cublas_version() >= 130200,
+  NVTE_CHECK(transformer_engine::cuda::cublas_version() >= 130200,


shouldn't there be more than one places to add this transformer_engine::?

pggPL · 2026-01-29T00:25:38Z

/te-ci

pggPL · 2026-01-29T16:33:53Z

/te-ci

Signed-off-by: Pawel Gadzinski <[email protected]>

greptile-apps

_{4 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

Signed-off-by: Pawel Gadzinski <[email protected]>

greptile-apps

_{2 files reviewed, 3 comments}

_{Edit Code Review Agent Settings | Greptile}

transformer_engine/common/gemm/cublaslt_grouped_gemm.cu

transformer_engine/common/include/transformer_engine/gemm.h

Signed-off-by: Pawel Gadzinski <[email protected]>

greptile-apps

_{2 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

transformer_engine/common/include/transformer_engine/gemm.h

Signed-off-by: Pawel Gadzinski <[email protected]>

greptile-apps

_{2 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

greptile-apps · 2026-01-30T16:51:08Z

transformer_engine/common/gemm/cublaslt_grouped_gemm.cu

                       cudaStream_t stream) {
-  NVTE_ERROR("nvte_grouped_gemm requires cuBLAS 13.1+, but compile-time cuBLAS version is ",
+  NVTE_ERROR("nvte_grouped_gemm requires cuBLAS 13.2+, but compile-time cuBLAS version is ",
             CUBLAS_VERSION, ". Please upgrade to CUDA 13.1 or newer.");


error message says "upgrade to CUDA 13.1 or newer" but should say "CUDA 13.2 or newer" to match the cuBLAS 13.2+ requirement

Suggested change

CUBLAS_VERSION, ". Please upgrade to CUDA 13.1 or newer.");

CUBLAS_VERSION, ". Please upgrade to CUDA 13.2 or newer.");

version change

77274c7

Signed-off-by: Pawel Gadzinski <[email protected]>

greptile-apps bot reviewed Jan 28, 2026

View reviewed changes

pggPL added the 2.12.0 label Jan 28, 2026

fix

e034376

Signed-off-by: Pawel Gadzinski <[email protected]>

greptile-apps bot reviewed Jan 29, 2026

View reviewed changes

zhongbozhu reviewed Jan 29, 2026

View reviewed changes

ifx

12f7e8f

Signed-off-by: Pawel Gadzinski <[email protected]>

greptile-apps bot reviewed Jan 29, 2026

View reviewed changes

fix

69773b3

Signed-off-by: Pawel Gadzinski <[email protected]>

greptile-apps bot reviewed Jan 29, 2026

View reviewed changes

transformer_engine/common/gemm/cublaslt_grouped_gemm.cu Outdated Show resolved Hide resolved

transformer_engine/common/include/transformer_engine/gemm.h Show resolved Hide resolved

transformer_engine/common/include/transformer_engine/gemm.h Show resolved Hide resolved

fix

bd5de11

Signed-off-by: Pawel Gadzinski <[email protected]>

greptile-apps bot reviewed Jan 29, 2026

View reviewed changes

transformer_engine/common/include/transformer_engine/gemm.h Show resolved Hide resolved

transformer_engine/common/include/transformer_engine/gemm.h Show resolved Hide resolved

fix

3ae26d3

Signed-off-by: Pawel Gadzinski <[email protected]>

greptile-apps bot reviewed Jan 30, 2026

View reviewed changes

ptrendx approved these changes Jan 30, 2026

View reviewed changes

pggPL merged commit c3769cb into NVIDIA:main Jan 30, 2026
10 of 12 checks passed

	CUBLAS_VERSION, ". Please upgrade to CUDA 13.1 or newer.");
	CUBLAS_VERSION, ". Please upgrade to CUDA 13.2 or newer.");

Fix minimum version of cublas for grouped gemm #2631

Fix minimum version of cublas for grouped gemm #2631

Uh oh!

Conversation

pggPL commented Jan 28, 2026

Description

Type of change

Checklist:

Uh oh!

pggPL commented Jan 28, 2026

Uh oh!

greptile-apps bot commented Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Overview

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Sequence Diagram

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

pggPL commented Jan 28, 2026

Uh oh!

pggPL commented Jan 29, 2026

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

zhongbozhu Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

pggPL commented Jan 29, 2026

Uh oh!

pggPL commented Jan 29, 2026

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

greptile-apps bot commented Jan 28, 2026 •

edited

Loading